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ABSTRACT 


This  report  presents  a  numerical  example  showing  how  selection  testing  can  reduce  costs  in  advanced  T-45 
training.  Selection  testing  provides  savings  by  reducing  the  rate  of  attrition.  With  reduced  attrition,  fewer  students 
are  required  to  enter  the  training  curriculum  to  achieve  a  set  number  of  graduates  to  fill  operational  seats. 

Balanced  against  this  positive  impact  are  at  least  three  factors  associated  with  selection  testing  that  must  be  taken 
into  account  in  any  cost-benefit  analysis.  These  are  1)  R&D  costs,  2)  administrative  costs,  and  3)  costs  associated 
with  rejecting  qualified  applicants.  It  is  shown  that,  under  certain  readily  quantifiable  circumstances,  selection 
testing  might  save  over  $2  million  per  T-45  advanced  strike  class.  The  major  goal  of  this  report  is  to  provide  a 
quantitative  framework  for  the  rational  discussion  of  selection  tests  as  a  prelude  and  partner  to  military  training 
curricula. 
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INTRODUCTION 


The  main  purpose  of  this  report  is  to  show  how  selection  testing  can  have  a  proven,  quantifiable  impact  on 
reducing  costs  for  a  specific  advanced  flight  training  pipeline.  The  argument  is  conducted  in  the  form  of  a 
numerical  example  using  actual  dollar  cost  data.  The  advanced  strike  curriculum  was  chosen  as  the  focus  for  this 
numerical  example  because  of  the  combination  of  high  percentage  of  failures  and  the  large  cost  of  each  failure. 
Conservative  cost  per  attrition  data  for  various  pipelines  are  presented  in  Table  1.  (Because  of  traditional  Navy 
usage,  “attrition”  and  “attrites”  are  used  interchangeably  for  “failure”  and  “a  student  who  has  failed.”)  These  cost 
data  are  not  exactly  correct,  but  they  reflect  the  general  relationship  among  the  various  pipelines.  This  is  sufficient 
for  the  numerical  examples  to  follow. 

Table  1.  The  Total  Cost  Due  to  Attrition  for  the  Various  Phases  of  Flight  Training. 


Source 

Number  of  failures 

Cost  per  failure  ($) 

Total  cost  ($) 

ocs 

90 

5,000.00 

450,000.00 

API 

27 

18,000.00 

486,000.00 

Primary 

79 

50,000.00 

3,950,000.00 

Intermediate  strike 

12 

350,000.00 

4,200,000.00 

Intermediate  maritime 

2 

100,000.00 

200,000.00 

Intermediate  helo 

4 

100,000.00 

400,000.00 

Intermediate  E2/C2 

1 

150,000.00 

150,000.00 

Adviced  strike 

i8 

500,000.00 

9,000,000.00 

Advanced  maritime 

4 

200,000.00 

800,000.00 

Advanced  helo 

12 

200,000.00 

2,400,000.00 

Advanced  E2/C2 

3 

500,000.00 

1,500,000.00 

Figure  1  presents  a  somewhat  idealized  “flow  chart”  describing  the  fate  of  1,000  naval  and  Marine  Corps 
aviation  students  as  they  progress  through  the  flight  training  curriculum.  The  numbers  contained  in  this  chart 
were  constructed  from  current  data  as  provided  by  the  Chief  of  Naval  Air  Training  (CNATRA)  Corpus  Christi, 

TX  (1).  The  numbers  deviate  slightly  from  the  actual  data  but  capture  the  essence  of  the  flow  and  attrition  of 
students  for  the  various  pipelines.  As  mentioned  above,  this  report  concentrates  on  the  advanced  strike  portion  of 
the  flow  chart  because  this  pipeline  is  the  most  expensive  in  terms  of  the  percentage  of  students  who  attrite  and  the 
cost  per  attrition. 

The  Navy  is  currently  transitioning  from  the  older  T-2  and  A-4  aircraft  in  the  advanced  strike  curriculum  to  the 
newer  T-45  aircraft.  Therefore,  during  this  transition  period,  the  attrition  rate  may  be  higher  than  normal  until  the 
training  regime  stabilizes  through  experience  with  the  T-45.  A  higher  attrition  rate  (8%)  than  previous  historical 
records  for  advanced  strike  is  inserted  into  the  flow  chart  for  this  reason. 

The  main  results  are  presented  in  the  next  section  in  the  form  of  a  spreadsheet  (Table  2).  This  analysis  shows 
that  over  $2.3  million  could  be  saved  if  a  selection  test  could  be  found  that  reduced  attrition  from  8%  to  5%.  In  the 
third  section,  a  more  detailed  explanation  is  provided  for  the  numbers  that  appear  in  the  spreadsheet  of  Table  2. 

Of  course,  the  numbers  used  in  this  example  merely  illustrate  the  type  of  analysis  that  can  be  carried  out,  and  better 
numbers,  if  forthcoming,  would  result  in  a  better  analysis.  Nonetheless,  we  believe  that  this  kind  of  cost-benefit 
analysis  is  generally  indicative  of  the  untapped  potential  of  selection  tests. 

The  numbers  given  in  Table  2  for  the  average  cost  of  the  status  quo  and  selection  testing  are  really  just  an 
approximation  based  on  obtaining  exactly  18  failures  in  the  status  quo  or  exactly  11  failures  for  selection  testing. 

In  fact,  there  is  a  probability  distribution  over  the  entire  number  of  possible  failures,  and  an  exact  analysis  would 
use  this  probability  distribution  to  calculate  an  average  cost  for  the  two  scenarios  being  compared. 
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Figure  1 .  A  slightly  idealized  **flow  chart"  showing  what  happens  to  1000  naval  and  Marine  Corps  aviation  students 
as  they  progress  through  training  from  Officer  Candidate  School  (OCS)  until  assignment  in  the  fleet. 

The  procedure  for  the  exact  calculation  of  the  average  cost  for  the  status  quo  and  selection  testing  is  presented 
in  the  last  section.  The  number  of  attritions  that  appear  in  the  top  part  of  the  spreadsheet  of  Table  2  are  just  the 
expected  number  of  attritions,  and  the  calculation  is  carried  out  with  this  number.  Compared  to  the  exact 
calculation,  the  approximation  used  in  the  spreadsheet  of  Table  2  is  fairly  accurate. 

A  discussion  of  the  cost-benefit  trade  off  in  advanced  flight  training  can  be  found  elsewhere  (2).  A  detailed 
mathematical  exposition  of  the  Bayesian  approach  for  the  predictive  distribution  used  for  calculating  the  exact  cost 
is  given  in  a  companion  paper  (3). 

SPREADSHEET  FOR  THE  NUMERICAL  EXAMPLE 

The  benefit  part  of  the  cost-benefit  analysis  is  presented  first.  This  shows  the  reduction  in  the  average  number 
of  attritions  for  a  specified  improvement  due  to  a  selection  test.  Given  the  cost  per  attrition,  the  overall  average 
cost  for  the  status  quo  can  be  calculated.  In  the  same  manner,  the  overall  average  cost  after  selection  testing  has 
been  implemented  is  calculated.  The  difference  between  these  two  numbers  is  defined  as  the  benefit 
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Table  2.  Spreadsheet  Illustrating  a  Cost-benefit  Analysis  for  the  Advanced  Strike  Pipeline. 


Number  in  training 

Attritions 

Cost  per  attrition  ($) 

Average  cost  ($) 

Status  quo 

218 

18 

500,000 

9,000,000 

Selection  testing 

211 

11 

500,000 

5,500,000 

Benefit 

3,500,000 

Cost  category 

Subjects 

Cost  per  subject  ($) 

Total  ($) 

R&D 

386 

1,000 

386,000 

Administrative 

272 

2,000 

544,000 

Rejection 

50 

5,000 

250,000 

jCost 

$1,180,000 

Benefit 

$3,500,000 

Cost 

$1,180,000 

1  Savings 

$2, 3,20, 000 

The  middle  of  the  spreadsheet  in  Table  2  presents  the  cost  part  of  the  cost-benefit  analysis.  Three  costs  are 
identified:  (1)  Research  and  Development  (R&D)  costs  associated  with  setting  up  an  experiment  to  test  a  sufficient 
number  of  subjects  to  determine  whether  the  mean  success  rate  has  been  raised,  (2)  routine  costs  of  administering 
an  operational  version  of  the  test  for  a  sufficient  number  of  candidates  to  meet  .fleet  requirements,  and  (3)  unique 
costs  associated  with  a  subjective  judgment  of  costs  incurred  from  candidates  rejected  by  the  selection  test  who 
otherwise  would  have  passed  the  training  curriculum. 

The  bottom  of  Table  2  shows  whether  the  BENEFITS  minus  the  COSTS  results  in  a  SAVINGS.  If  the 
SAVINGS  is  above  some  reasonable  threshold,  then  an  argument  can  be  made  to  support  funding  for  selection 
testing  R&D.  In  this  numerical  example,  a  SAVINGS  of  $2,320,000  was  realized. 

EXPLANATION  OF  NUMBERS  IN  THE  SPREADSHEET 
Benefit 

For  the  sake  of  this  numerical  example,  assume  that  selection  testing  can  reduce  the  rate  of  attrition  from  the 
current  8%  down  to  5%.  We  assume  that  a  sufficient  number  of  students  are  brought  into  the  pipeline  so  that  the 
number  of  trained  aviators  required  for  fleet  assignment  is  at  the  mean  of  the  distribution.  For  this  example, 
inspection  of  Fig.  1  reveals  that  200  students  are  needed  to  fill  the  operational  seats  for  the  strike  community. 
Therefore,  if  218  students  are  admitted  into  the  pipeline  under  the  status  quo  of  8%  attrition,  200  students  will  be 
the  mean  number  graduated.  There  will  be,  on  the  average,  18  attritions  at  the  given  cost  of  $500,000  per  attrite. 

This  situation  is  contrasted  with  the  selection  testing  scenario  where  attrition  has  been  reduced  to  5%. .  Now  only 
21 1  students  need  to  be  admitted  into  the  training  curriculum  to  obtain  an  average  graduation  of  200  students. 

This  difference  of  7  attritions  between  the  status  quo  and  selection  testing  results  in  a  benefit  of  about  $3.5 
million. 
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Selection  testing  entails  certain  costs  that  are  not  present  under  the  status  quo.  These  must  be  subtracted  from  the 
benefit  due  to  selection  testing.  First,  there  are  the  costs  of  designing  experiments,  running  subjects,  and  analyzing 
data.  The  number  of  subjects  tested  must  be  large  enough  so  that  a  reasonable  estimate  of  the  new  (hopefully  higher) 
success  rate  can  be  obtained. 

The  numbers  that  make  up  these  R&D  costs  are  more  readily  understood  by  viewing  Fig.  2.  Imagine  a  normal 
distribution  of  selection  test  scores  for  those  subjects  that  eventually  fail  the  training  curriculum.  This  normal  curve 
is  shown  drawn  around  a  mean  of  0  with  a  standard  deviation  of  1.  Therefore,  99%  of  the  selection  test  scores 
obtained  by  these  students  who  failed  training  range  from  -3  to  +3. 


ProbabiliV 

Densi^ 


Fail  Pass  PREDICTED 


356 

30 

386 


Figure  2.  A  sketch  of  two  normal  distribution  functions,  one  for  scores  from  subjects  who  eventually  failed  training, 
the  other  from  subjects  who  passed  training.  The  two  distributions  are  separated  by  .84  SD  where  each  distribution 
has  a  SD  ofl.  The  cut-off  score  is  placed  at  the  mean  of  the  fail  group.  The  table  at  the  right  summarizes  the 
numbers  of  subjects  in  various  categories. 

A  line  is  drawn  at  the  mean  of  these  scores  (i.e.,  at  a  score  of  0).  This  line  divides  the  normal  curve  in  half  with 
50%  of  the  failed  students  to  the  right  half  of  this  line  and  50%  of  the  failed  students  to  the  left  half  of  the  line.  If 
there  were  30  students  in  this  group  that  failed,  then  15  students  would  have  obtained  a  selection  score  greater  than 
0,  and  15  students  would  have  obtained  a  selection  test  score  less  than  0.  This  line  erected  at  the  mean  of  the 
distribution  of  selection  test  scores  for  the  FAIL  group  is  called  a  threshold,  or,  cut-off  score. 


Now  imagine  a  second  normal  distribution  of  selection  test  scores  for  the  subjects  who  eventually  passed  training. 
If  the  selection  test  is  at  all  helpful,  the  mean  of  this  distribution  for  the  PASS  group  will  be  higher  than  the  mean  of 
selection  test  scores  for  the  FAIL  group.  For  this  numerical  example,  we  want  to  place  the  mean  of  scores  for  the 
PASS  group  such  that  the  threshold  score  divides  the  PASS  normal  curve  into  two  sections,  just  as  it  did  for  the 
FAIT,  group.  We  want  the  section  to  the  left  of  the  threshold  score  to  contain  20%  of  the  number  of  students  who 
passed  and  the  complementary  section  to  the  right  of  the  threshold  score  to  contain  the  other  80%  of  the  students 
who  passed. 
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By  consulting  a  table  of  the  normal  distribution,  one  finds  a  score  that  is  .84  standard  deviations  below  the  mean 
of  the  PASS  group  divides  off  the  required  20%.  Since  the  scores  for  both  the  FAIL  and  the  PASS  group  have  a 
standard  deviation  of  1,  the  mean  of  the  selection  test  scores  for  the  students  who  passed  training  exists  at  +.84. 
Therefore,  99%  of  the  selection  test  scores  for  the  students  who  passed  training  ranges  from  +3.84  to  -2,16. 

If  there  were  a  total  of  356  students  in  the  PASS  group,  then  requiring  80%  on  the  right-hand  side  of  the  cut-off 
score  results  in  285  students  (to  the  nearest  integer)  with  the  remaining  71  students  representing  the  20%  to  the  left 
of  the  threshold  score.  We  have  already  calculated  that  15  students  exist  in  both  the  left  and  right  sections  of  the 
normal  curve  for  the  FAIL  group. 

If  we  PREDICT  PASS  for  any  score  that  exceeds  the  cut-off  score  and  PREDICT  FAIL  for  any  score  that  falls 
below  the  cut-off  score,  then  there  are  two  ways  to  be  correct  and  two  ways  to  be  wrong  in  the  prediction.  This  is  all 
conveniently  summarized  in  the  2  x  2  table  shown  to  the  right  of  the  two  normal  curves  in  Fig.  2.  The  total  number 
of  PREDICTED  PASSES  will  be  300,  of  which  285  will  actually  PASS,  resulting  in  the  5%  attrition  rate  that  we 
were  seeking  for  this  selection  test.  Taking  into  account  the  PREDICTED  FAILS,  there  will  be,  in  all,  a  total  of  386 
subjects  run  in  the  experiment,  and  the  estimated  cost  for  this  is  shown  in  the  cost-benefit  spreadsheet. 

This  figure  for  R&D  costs  is  actually  quite  conservative.  Normally,  we  would  prorate,  or  amortize,  the  cost  of  the 
R&D  over  the  life  of  the  test,  which  would  lower  this  particular  cost.  However,  we  retain  this  non-amortized  cost 
figure  as  a  worst-case  scenario. 

The  above  explanation  is  entirely  analogous  to  Signal  Detection  Theory  (SDT).  The  separation  between  the 
means  (d’  in  SDT)  indicates  how  well  the  test  battery  correlates  with  training  performance.  The  placement  of  the  cut¬ 
off  score  (6  in  SDT)  indicates  the  influence  of  prior  probabilities  and  the  values  given  to  making  correct  and 
incorrect  decisions. 

The  selection  test  scores  that  are  normally  distributed  in  Fig.  2  are  composite  scores  derived  firom  individual  test 
scores  making  up  the  test  battery.  Such  a  composite  score  that  best  separates  two  groups  like  the  PASS  and  FAIL 
groups  might  come  from  a  statistical  technique  like  discriminant  analysis. 

Administrative  Costs 

Assuming  success  in  the  experimental  phase  of  selection  testing  leads  to  the  question  of  an  operational 
implementation  of  the  selection  test.  How  many  candidates  need  to  be  tested  on  a  recurring  basis  should  the 
selection  test  actually  be  used  as  a  screening  device?  The  same  line  of  reasoning  used  above  to  determine  the 
number  of  subjects  in  the  R&D  phase  can  also  be  employed  to  determine  the  total  number  of  candidates  that  need  to 
be  tested  during  routine  administration  of  the  selection  test  battery. 

If  the  selection  test  battery  has  achieved  the  new  lower  5%  attrition  rate,  then  21 1  subjects  are  going  to  be 
admitted  into  the  advanced  T-45  curriculum.  Given  the  argument  encapsulated  in  the  2  x  2  table  of  Fig.  2,  we  know 
that  200  are  predicted  passes  and  1 1  are  the  expected  attritions.  During  this  routine  administrative  testing,  50 
candidates  will  be  declared  failures  from  those  that  would  have  passed  (200  is  80%  of  250)  and  1 1  other  candidates 
will  be  correctly  predicted  failures.  Thus  we  arrive  at  a  total  of  272  candidates  who  would  have  to  be  tested  on  a 
recurring  basis  to  provide  the  200  expected  successes  out  of  21 1  trainees.  Operational  testing  requires  some  fixed 
costs  for  employees,  testing  space,  maintenance,  updating,  et  cetera,  so  we  assign  a  value  of  $2,0(X)  per  candidate 
tested  to  arrive  at  this  figure. 

Rejection  Costs 

It  is  difficult  via  the  selection  model  to  assign  a  cost  of  rejecting  50  candidates  who  otherwise  would  have 
succeeded.  We  arbitrarily  assign  a  cost  of  $5,000  per  student  to  reflect  the  fact  that  some  attention  should  be  paid  to 
this  trade  off  in  the  cost-benefit  analysis. 
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Given  the  circumstances  in  this  example,  it  seems  that  over  $2  million  could  be  saved  on  a  recurring  basis  through 
the  use  of  selection  testing  specifically  targeted  to  identify  attritions  in  the  advanced  phase  of  T-45  training.  Similar 
arguments  could  be  made  for  other  pipelines,  although  the  savings  would  be  less  because  of  a  lower  percentage  of 
failures  and/or  the  lower  cost  of  attritions. 

The  Exact  Calculation  of  the  Average  Cost 

The  average  cost  for  the  status  quo  versus  selection  testing  as  given  in  the  spreadsheet  of  Table  2  is  an 
approximation  based  on  the  expected  number  of  attritions  for  these  two  situations.  The  actual  number  of  graduates, 
and,  therefore  the  number  of  attritions,  will  fluctuate  from  class  to  class  reflecting  the  underlying  probabilistic  nature 
of  success  in  the  training  curriculum.  We  first  present  the  predictive  probability  distribution  for  the  status  quo  where 
the  data  indicate  an  attrition  rate  of  8%  and  then  compare  Ais  with  the  predictive  probability  distribution  for 
selection  testing  where  the  data  from  the  experiment  indicate  a  lowering  of  the  attrition  rate  to  5%.  Having  these 
probability  distributions  in  hand  makes  it  an  easy  matter  to  calculate  the  average  cost  for  each  scenario. 

Figure  3  shows  the  probability  distribution  for  the  status  quo  where  no  specialized  selection  test  is  used  to  target 
advanced  T-45  failures.  The  actual  probability  distribution  should  cover  the  entire  range  of  possible  students 
graduated,  i.e.,  from  0  to  218.  However,  since  the  probability  of  a  small  number  of  graduates  is  so  extremely  close 
to  zero  for  an  attrition  rate  known  from  actual  data  to  be  around  8%,  only  a  portion  of  the  discrete  predictive 
probability  distribution  is  presented  in  Fig.  3.  The  number  of  graduates  along  the  x  axis  ranges  from  148  to  218,  the 
maximum  number  of  graduates  possible  from  a  class  of  218.  This  distribution  peaks  at  201  graduates  (the  mode) 
and  then  falls  off  in  a  gradual  manner  for  less  or  more  graduates.  Because  there  is  a  upper  wall  at  218  students,  these 
distributions  are  slightly  skewed. 
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Figure  3.  The  discrete  predictive  probability  distribution  for  the  number  of  graduates  in  the  advanced  T-45 
curriculum  based  on  a  supposition  of  8%  attrition.  This  attrition  rate  is  the  usual  rate  without  using  any 
specialized  selection  test.  The  x  axis  consists  of  the  number  of  graduates  from  a  class  size  of  21 8.  The  y  axis  gives 
the  probability  for  the  specified  number  of  graduates. 

Perusing  Fig.  3  reveals  a  realistic  probability  that  a  training  class  might  graduate,  instead  of  the  200  students  near 
the  mode  of  the  distribution,  only  196  students  for  a  total  of  22  attritions.  On  the  other  hand,  208  students  might 
graduate  in  another  class,  leading  to  only  10  attritions.  The  technical  and  computational  details  of  deriving  these 
predictive  probability  distributions  are  presented  in  Blower  (3).  For  a  more  fundamental  expository  treatment  of  the 
Bayesian  predictive  probability  distribution  see  Blower  (4). 
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Figure  4  shows  the  comparable  graph  for  the  probability  of  students  graduating  given  a  new  attrition  rate  of  5%. 
The  X  axis  now  extends  only  to  21 1  students  because  this  is  the  new  maximum  class  size  allowable  from  the  lower 
attrition  rate.  Although  the  graph  still  peaks  at  201  students,  there  is  some  appreciable  spread  to  fewer  numbers 
graduating.  This  results  from  the  fact  diat  the  curve  is  constructed  from  a  smaller  sample  size,  namely  the  300 
students  used  in  the  experiment.  A  smaller  sample  size  results  in  more  variability  of  the  predictive  probability 
distribution. 


Probability 


Figure  4.  The  discrete  predictive  probability  distribution  for  the  number  of  graduates  in  the  advanced  T-45 
curriculum  based  on  a  supposition  of  5%  attrition  by  using  a  selection  test.  The  x  axis  consists  of  the  number  of 
graduates  from  a  class  size  of  211.  The  y  axis  gives  the  probability  for  the  specified  number  of  graduates. 

The  exact  calculation  of  the  average  cost  is  accomplished  by  taking  the  sum  over  all  possible  graduates  (in  the 
first  example  from  0  to  218)  of  the  cost  of  that  number  of  students  attriting  times  the  probability  of  that  number  of 
students  attriting.  This  computation  is  illustrated  in  Table  3,  which  looks  at  the  range  of  180  to  218  students 
graduating,  or,  in  other  words,  from  0  to  38  attritions. 

For  example,  the  first  row  shows  the  relevant  information  for  the  situation  where  180  students  graduate.  From  a 
class  of  218  students,  if  180  students  graduate,  then  38  must  have  attrited  as  shown  in  the  second  column.  The 
probability  of  38  students  attriting  is  taken  from  the  discrete  predictive  probability  distribution  as  shown  in  the  third 
column.  The  chance  of  38  students  attriting  is  very  small,  but  there  is  a  large  cost  of  $19  million  attached  to  this 
number  of  attritions  as  shown  in  column  four.  The  last  column  shows  the  multiplication  of  the  probability  of  38 
students  attriting  times  the  cost  of  38  students  attriting.  This  computes  to  $492.60.  This  is  the  contribution  to  the 
overall  sum  for  this  particular  number  (38)  students  failing. 

We  began  Table  3  where  the  number  of  attritions  started  to  have  some  monetary  impact  on  the  average  cost.  The 
succeeding  rows  in  Table  3  carry  out  this  same  computation  as  just  described  for  the  first  row  for  each  number  of 
students  up  to  a  maximum  class  size  of  218.  When  the  last  column  is  summed,  this  is  the  average  cost  for  an  8% 
attrition  rate.  The  exact  value  of  the  average  cost  as  computed  over  the  entire  range  from  0  to  218  students  is 
$8,855,210.20. 
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Table  3.  Detailed  Calculation  of  the  Overall  Cost  due  to  Attrition  When  the  Status  Quo  is  Maintained.  The 
Probability  for  Each  Number  of  Students  Failing  Comes  From  the  Predictive  Probability  Density  Function  as 
Illustrated  in  Fig.  3. 


Number  of  students 
graduating 

Number  of 
students  failing 

Probability  of  this 
number  failing 

Cost  for  this  number 
failing  ($) 

Overall  contribution 
to  cost  ($) 

180 

38 

0.0000259 

19,000,000 

492.60 

181 

37 

0.0000508 

18,500,000 

939.60 

182 

36 

0.0000973 

18,000,000 

1,750.62 

183 

35 

0.0001819 

17,500,000 

3,183.83 

184 

34 

0.0003322 

17,000,000 

5,648.19 

185 

33 

0.0005919 

16,500,000 

9,766.47 

186 

32 

0.0010279 

16,000,000 

16,446.81 

187 

31 

0.0017387 

15,500,000 

26,950.32 

188 

30 

0.0028621 

15,000,000 

42,931.79 

189 

29 

0.0045806 

14,500,000 

66,418.89 

190 

28 

0.0071204 

14,000,000 

99,685.75 

191 

27 

0.0107390 

13,500,000 

144,976.47 

192 

26 

0.0156962 

13,000,000 

204,050.68 

193 

25 

0.0222050 

12,500,000 

277,562.97 

194 

24 

0.0303628 

12,000,000 

364,353.53 

195 

23 

0.0400702 

11,500,000 

460,806.91 

196 

22 

0.0509550 

11,000,000 

560,505.29 

197 

21 

0.0623262 

10,500,000 

654,424.70 

198 

20 

0.0731858 

10,000,000 

731,857.59 

199 

19 

0.0823236 

9,500,000 

782,074.03 

200 

18 

0.0884977 

9,000,000 

796,479.09 

201 

17 

0.0906779 

8,500,000 

770,762.07 

202 

16 

0.08829*75 

8,000,000 

706,379.90 

203 

15 

0.0814379 

7,500,000 

610,784.47 

204 

14 

0.0708758 

7,000,000 

496,130.49 

205 

13 

0.0579549 

6,500,000 

376,707.14 

206 

12 

0.0443046 

6,000,000 

265,827.62 

207 

11 

0.0314822 

5,500,000 

173,151.94 

208 

10 

0.0206531 

5,000,000 

103,265.49 

209 

9 

0.0124076 

4,500,000 

55,834.06 

210 

8 

0.0067592 

4,000,000 

27,036.78 

211 

7 

0.0032985 

3,500,000 

11,544.90 

212 

6 

0.0014199 

3,000,000 

4,259.77 

213 

5 

0.0005284 

2,500,000 

1,321.01 

214 

4 

0.0001654 

2,000,000 

330.78 

215 

3 

0.0000419 

1,500,000 

62.79 

216 

2 

0.0000081 

1,000,000 

8.07 

217 

1 

0.0000011 

500,000 

0.54 

218 

0 

0.0000000 

0 

0.00 

Cumulative  probability  .9999999  Average  cost  $  8,855,210,20 


A  similar  exercise  can  be  conducted  if  selection  testing  has  been  implemented.  The  details  are  given  in  Table 
4.  In  this  case,  the  average  cost  for  a  5%  attrition  rate  is  $5,645,397.76  resulting  in  a  benefit  of  $3,209,812.44  for 
selection  testing  over  the  status  quo,  somewhat  less  than  the  $3,500,000  as  calculated  in  the  spreadsheet.  This  cost 
figure,  of  course,  represents  the  savings  before  the  unique  costs  associated  with  selection  testing  are  factored  in. 
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When  the  actual  average  cost  figures  as  just  computed  are  inserted  into  Table  2,  the  overall  savings  due  to  selection 
testing  is  $2,029,812.44.  The  approximation  given  by  the  expected  number  of  attritions  of  18  for  the  status  quo 
and  1 1  for  selection  testing  gives  a  reasonable  approximation  to  the  exact  calculated  benefit  as  performed  in  this 
section. 

It  is  important  to  remember  that,  given  the  Bayesian  treatment  of  this  problem,  there  is  a  probability  density 
function  associated  with  both  the  attrition  rate  for  the  status  quo  and  the  attrition  rate  for  selection  testing.  These 
functions  are  centered  around  8%  for  the  status  quo  and  5%  for  selection  testing,  but  also  allow  for  the  possibility 
of  lower  and  higher  rates.  So  the  8%  and  5%  values  are  convenient  labels  locating  the  centers  of  these 
distributions  and  permit  a  rough  approximation  such  as  carried  out  in  Table  2.  It  would  be  wrong  to  think  of  these 
two  values  as  fixed  and  unvarying  with  no  attached  uncertainty.  The  exact  calculation,  which  does  take  account  of 
the  uncertainty  surrounding  the  8%  and  5%  attrition  rates  through  the  predictive  distribution,  was  illustrated  in 
this  section.  Again,  this  is  all  spelled  out  in  greater  detail  in  the  companion  report  (3). 
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Table  4.  Detailed  calculation  of  the  overall  cost  due  to  attrition  when  selection  testing  is  employed.  The 
probability  for  each  number  of  students  failing  comes  from  the  predictive  probability  density  function  as  illustrated 
in  Fig.  4. 


Number  of  students 
graduating 

Number  of 
students  failing 

Probability  of  this 
number  failing 

Cost  for  this  number 
failing  ($) 

Overall  contribution 
to  cost  ($) 

173 

38 

0.0000010 

19,000,000 

19.84 

174 

37 

0.0000020 

18,500,000 

36.55 

175 

36 

0.0000037 

18,000,000 

66.51 

176 

35 

0.0000068 

17,500,000 

119.56 

177 

34 

0.0000125 

17,000,000 

212.21 

178 

33 

0.0000225 

16,500,000 

371.75 

179 

32 

0.0000402 

16,000,000 

642.46 

180 

31 

0.0000706 

15,500,000 

1,094.74 

181 

30 

0.0001225 

15,000,000 

1,838.24 

182 

29 

0.0002096 

14,500,000 

3,039.87 

183 

28 

0.0003534 

14,000,000 

4,947.43 

184 

27 

0.0005866 

13,500,000 

7,918.76 

185 

26 

0.0009581 

13,000,000 

12,454.81 

186 

25 

0.0015386 

12,500,000 

19,232.41 

187 

24 

0.0024274 

12,000,000 

29,128.74 

188 

23 

0.0037586 

11,500,000 

43,224.35 

189 

22 

0.0057061 

11,000,000 

62,766.66 

190 

21 

0.0084830 

10,500,000 

89,071.11 

191 

20 

0.0123337 

10,000,000 

123,337.33 

192 

19 

0.0175121 

9,500,000 

166,364.72 

193 

18 

0.0242413 

9,000,000 

218,171.55 

194 

17 

0.0326536^ 

8,500,000 

277,555.63 

195 

16 

0.0427103* 

8,000,000 

341,682.24 

196 

15 

0.0541116 

7,500,000 

405,837.34 

197 

14 

0.0662173 

7,000,000 

463,521.07 

198 

13 

0.0780076 

6,500,000 

507,049.29 

199 

12 

0.0881247 

6,000,000 

528,748.20 

200 

11 

0.0950273 

5,500,000 

522,650.33 

201 

10 

0.0972703 

5,000,000 

486,351.64 

202 

9 

0.0938769 

4,500,000 

422,445.91 

203 

8 

0.0847129 

4,000,000 

338,851.43 

204 

7 

0.0707234 

3,500,000 

247,531.86 

205 

6 

0.0538842 

3,000,000 

161,652.53 

206 

5 

0.0367894 

2,500,000 

91,973.56 

207 

4 

0.0219464 

2,000,000 

43,892.71 

208 

3 

0.0110239 

1,500,000 

16,535.92 

209 

2 

0.0044013 

1,000,000 

4,401.30 

210 

1 

0.0012696 

500,000 

634.80 

211 

0 

0.000000 

0 

0.00 

Cumulative  Probability 

.9999999 

Average  Cost 

$  5,645,397.76 
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